Conventional resource sharing platforms allow frameworks, e.g., applications, to choose to accept or decline resource offers made by a master based on available resources in slaves. If a framework accepts a resource offer, those resources are given to that framework for some period of time. The framework scheduler then assigns workload in the form of tasks to be run on a slave for the accepted offer. When the tasks are completed, the freed up resources are returned so they can be offered to other frameworks.
One difficulty in building a framework in conventional resource sharing platforms is creating logic to determine which resource offers to accept and which offers to decline. For example, an offer may be suitable for present framework needs, but a better resource offer may be coming soon, such as within a few seconds. Perhaps the present offer is as good an offer as the framework can expect to receive for some time. In general, a conventional framework cannot predict whether better resource offers will be received.
Furthermore, in conventional resource sharing platforms the framework has no idea on the quality of the resource offer. Even with the resource partitioning of Linux cGroups, for example a slice of a CPU on a lightly loaded slave machine might have better expected throughput than a slice of a CPU on a slave machine running many tasks. The throughput of disk I/O on one slave machine may be much better than can be expected on a different slave machine. Conventional frameworks lack suitable logic for choosing between an offer from one slave machine and an equivalent offer from another slave machine. Furthermore, information changes over time as workload increases and decreases across the collection of slave machines in the cluster.